Microscopic diagonal entropy and its connection to basic thermodynamic relations 
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We define a diagonal entropy (d-entropy) for an arbitrary Hamiltonian system as Sd = — 
53 /°nn In Pnn with the sum taken over the basis of instantaneous energy states. In equilibrium this 
entropy coincides with the conventional von Neumann entropy S n = — Tr pin p. However, in contrast 
to S n , the d-entropy is not conserved in time. If the system is initially in thermal equilibrium 
characterized by temperature T then in accord with the second law of thermodynamics the d- 
entropy can only increase or stay the same, the d-entropy also automatically satisfies fundamental 
thermodynamic relation, which reduces to the first law of thermodynamics for quasi-stationary 
processes. The d-entropy is also automatically conserved for adiabatic processes. We illustrate 
our results with explicit examples and show that its behavior is consistent with expectations from 
thermodynamics. 



Explicit relations between thermodynamical and mi- 
croscopical quantities have attracted attention of physics 
community for a long time One of the main chal- 
lenges is the correct microscopic definition of the entropy. 
While it is established that the von Neumann entropy 
S n = — Trplnp correctly describes thermal equilibrium, 
this entropy clearly disagrees the second law of thermo- 
dynamics because for isolated systems it is conserved in 
time Q. This entropy also does not satisfy the first law 
of thermodynamics or fundamental thermodynamic rela- 
tion unless one makes an additional assumption that the 
density matrix always remains diagonal. It is well under- 
stood that the thermodynamic entropy of a closed sys- 
tem obtained from the von Neumann entropy by coarse- 
graining increases with time (see E.g. Refs. 0, Q). If 
properly defined coarse-graining can lead to correct pre- 
dictions in complex systems subject to dynamical pro- 
cesses (see e.g. Refs. Yet the whole situation that 
on top of the microscopic description one needs to intro- 
duce a non-uniquely defined coarse-graining procedure, 
which is not part of the Hamiltonian dynamics, is not 
satisfactory. 

Microscopic description of thermodynamics using the 
von Neumann entropy is problematic on several other 
reasons. For example, consider a sufficiently complex 
system that was subject to a process which started and 
ended in a distant past, and eventually achieved some 
steady state. By the ergodic hypothesis time average of 
any thermodynamic observable should be equivalent to 
the equilibrium ensemble average. For any observable 
f2 its time average can be written as fl = Y] n Onnpnn; 
where p nn are the time independent diagonal elements of 
the density matrix in the eigenbasis of the Hamiltonian 
and £l nn are the diagonal matrix elements of the oper- 
ator Q. All information about arbitrary time averaged 
observables and thus about the steadystate is contained 
in diagonal elements of p (see Refs. H, 0] for more dis- 
cussion). At the same time the von Neumann entropy is 
not an observable. It explicitly depends on the nonlinear 
combination of the off-diagonal elements of the density 
matrix, which do not average to zero. So we have the 



situation that (i) the von Neumann entropy contains ad- 
ditional information, which does not appear in any ther- 
modynamic measurement and (ii) its time average is dif- 
ferent from the entropy of the equilibrium ensemble. 

There is another important indication that the entropy 
should be sensitive only to the diagonal matrix elements 
of pit). It follows from the basic thermodynamics that 
in adiabatic processes the entropy of the system does not 
change 2]. At the same time the slow changes in the 
Hamiltonian do not induce transitions between instan- 
taneous energy levels which implies that diagonal 
elements of the density matrix (and only they) do not 
change in time. We note that it is practically impossible 
to completely avoid transitions between energy levels in 
macroscopic systems [H, The proper adiabatic limit, 
therefore, should be defined as such when the heat gen- 
erated during a dynamical process is small. In turn heat- 
ing of the system is related to the transitions between 
different instantaneous energy levels @, [nj and is again 
sensitive only to the diagonal matrix elements of pit). 

These considerations suggest a simple resolution. Let 
us define the following diagonal entropy (or simply d- 
entropy) : 



Sd = - 2J Pnn In Pn 



(1) 



There is some ambiguity in this definition associated with 
possible degeneracies in the energy spectrum which is not 
important for the purposes of this work. If the Hamil- 
tonian is time-independent then Sd formally corresponds 
to the usual von Neumann entropy defined by the time- 
averaged density matrix p = limt_>oo 1/t J pit')dt' (see 
e.g. Ref. [3[). At the same time, the expression (JTJ) is de- 
fined instantaneously and thus does not require coarse- 
graining of the density matrix or the assumption that 
the Hamiltonian is stationary. The definition ([lj sug- 
gests a classical generalization where instead of a sum 
over many-body states one takes the integral over orbits 
corresponding to different energies of the system. The 
analogue of p nn would be a probability to find the system 
in the corresponding orbit: p{E) = J dxdp p(x, p)6(E — 
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E(x, p)), where x, p spans the many-particle phase space 
and p(x, p) is the probability to occupy a particular 
phase-space point. So the classical diagonal entropy 
can be defined as Sf = - J dEN{E)p{E) lnp(E), where 
N(E) = J dxdp5(E — E(x,p)) is the (many-particle) 
density of states. 

Let us show that Sd is consistent with the known prop- 
erties of the thermodynamic entropy. In equilibrium, 
when the density matrix is stationary (diagonal) , clearly 
Sd = S n . Thus Sd satisfies such requirements as exten- 
sivity, positivity. It also automatically vanishes in the 
zero-temperature limit satisfying the third law of ther- 
modynamics. Then we observe that the entropy Sd can 
change in time. If the initial state is stationary then 
for any time-dependent process we have Sd(t) > Sd(0). 
The proof of this statement can be adopted from Ref. [3j 
by straightforward generalization and we sketch it be- 
low. The d-entropy also satisfies a much stronger con- 
straint imposed by the fundamental thermodynamic re- 
lation (which is equivalent to the first law of thermo- 
dynamics for quasi-stationary processes). To be more 
specific we assume that initially a system is prepared in 
the Gibbs ensemble in which the density matrix is diago- 
nal and p° tx exp[— j3E n ]. The system is then perturbed 
either by changing external parameters Xj such as the 
system size or various couplings of the Hamiltonian, or 
by putting it into a contact with another system or both. 

Without specifying the details of the process we know 
that according to the fundamental thermodynamics rela- 
tion we must have 



AE w TAS - 



VI — 



AX j 



(2) 



where S is a proper thermodynamic entropy of the system 
and (dE/dXj) s are generalized forces [HHl- The equality 
^ should be always satisfied as long as AE, AS and AXj 
are infinitcsimally small, and we can neglect the higher- 
order differentials. 

Microscopically Eq. Q is a very nontrivial statement. 
In particular, assuming for simplicity a cyclic process 
such that all AXj = 0, this relation implies that the 
changes in the two microscopically defined quantities E 
and S both depending on the details of the process are 
proportional to each other with the proportionality con- 
stant being independent of these details. Let us show 
that the introduced d-entropy satisfies the equality @ 
automatically. First we write the change of the energy 
to the linear order in Ap and AXj\ 

AE w Y, AE n(t)Pn + E E„(0)Ap nn (t), (3) 



where AE n (t) — E n (t) — E n (0) is a change of the instan- 
taneous energy levels due to time evolution, Ap nn {t) is 
the change of the diagonal matrix elements of the den- 
sity matrix, and /9„ = p rm (0). In the adiabatic limit 



Ap nn (t) — and thus the first term in Eq. ([3]) corre- 
sponds to the adiabatic change of the energy AE^ft) 
while the second one corresponds to the heat Next 
we consider a similar expression for the change of the 
d-entropy. To the leading order in Ap nn (t) (using that 
Y, n Ap nn (t) = 0) we find 



AS d K-J2 A Pnn(t) lnp° n . 



(4) 



In thermal equilibrium we have p° cx exp[— (3E n (0)], and 
thus comparing Eqs. (|3|) and (j4]) we immediately find 



AE « AE : 



ad 



TAS d 



(5) 



The first term here AE^ is a function of the state, i.e. 
it depends only on the instantaneous values of the exter- 
nal parameters Xj and the initial probabilities Thus 
it can be expressed as AE a d = J2j(9E /dXj)s d AXj , and 
Eq. ([5]) is equivalent to the relation $2$. Note that the 
derivation of Eq. |(5J) required neither that the initial den- 
sity matrix is diagonal nor that the system was closed 
during the process. We remark that the d-entropy which 
depends only on the diagonal elements of the density ma- 
trix can be in principle determined from the distribution 
function of energy in the system or equivalently from the 
distribution of work done on the system to perform some 
dynamical process [Til ] . 

Properties of the d-entropy for closed systems. For 
closed Hamiltonian systems it is possible to make a fur- 
ther step and show that the d-entropy agrees with the 
second law of thermodynamics. Thus in Refs. [H, [l3| 
(see also Ref. [l(J) it was proven that if the initial den- 
sity matrix satisfies the conditions of passivity, i.e. it 
commutes with the Hamiltonian and its elements are 
monotonically decreasing functions of ener gy: (E n — 
E m )(Pn — Pm) < then in any cyclic process [23| the en- 
ergy of the system can either increase or stay the same: 
Q(t) = Y^ n E n Ap nn (t) > 0. This statement is equiva- 
lent to the second law of thermodynamics in the Kelvin's 
(Thompson's) form. 

Let us now rewrite the expression ([1]) in the following 
form: 



s d (t) = S d (0) - 



Ap nn (t) lnp° 



+(p° n + A Pnn {t)) In (1 + Ap nn (t)/p° n ) 



(6) 



The second term in this sum ASi is linear in Ap. Because 
lnp° always satisfies (p° - p° m ){\np Q n - \np° m ) > the 
proof of Refs. [L2J, [l3( immediately yields ASi > for 
any stationary initial p. The third term in Eq. ([6]) is non- 
positive and contains only quadratic and higher order 
terms in Ap. Therefore, we obtain an upper boundary 
estimate on d-entropy, < ASd(t) < ASi(t). In the 
linear order in Ap nn — > we have ASd(t) ~ ASi(t). 
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The fact that ASd(t) > in closed systems can be 
proven beyond the linear order in Ap as long as the ini- 
tial state is stationary. The proof is essentially adopted 
from Ref. (see Eqs. (2.4) - (2.10)) to the quantum 
case where instead of the time averaging we use the mi- 
croscopic definition of Sd(t). First, we identify p d (t) as a 
diagonal part of the full time-dependent density matrix 
P(t), i-e- pin(t) = Pnn{t) and p d nrn {t) = for n ^ m. 
By using the Jensen's inequality we obtain that (see also 
Eq. (1.40) in Ref. 0) 

Tr(-p In p + p d In p d ) < Tr(p - p d ) In p d = (7) 

Hence we find that S d (t) > S n {t). At the same time 
the von Neumann entropy is conserved in time, S n (t) = 
S n (0). Noting that S d (0) = S n (0), since the initial den- 
sity matrix is diagonal by the assumption, we prove 

S d (t) > S d (0). (8) 

Note that Eq. (|8|) does not imply that S d (t) is necessarily 
a monotonic function of time. 

Another important property of d-entropy is that it 
is not additive away from equilibrium. This is clear 
from Eq. ([5]), which implies that e.g. in closed sys- 
tems d-entropy can change only while external param- 
eters are changing in time. For example for sudden in- 
stantaneous quenches d-entropy also changes instanta- 
neously. At the same time equilibration of the system 
after the quench may take a long time during which 
there is energy exchange between different parts of the 
system. This implies that local d-entropies change on 
much longer time scales and during intermediate times 
the d-entropy is not additive. We will illustrate this non- 
additivity using an explicit example below. Note that 
in this respect the behavior of d-entropy is again sim- 
ilar to the behavior of the thermodynamic entropy de- 
fined as S = lnfi(.E), where for a microcanonical il(E) 
is the number of available states within a small energy 
interval SE (see e.g. Ref. [15]). In fact S d coincides 
with this entropy if we assume that p nn is constant inde- 
pendent of n within this energy window. Like S d , the 
entropy S defined in this way can change only if we 
change energy by heating the system or changing the 
external parameter or if we change density of states by 
changing the external parameter or both. In equilibrium 
the entropy S is additive. I.e. if we split the system 
into two weakly interacting subsystems 1 and 2 so that 
E = E 1 +E 2 then fl(E) « VL 1 (E 1 )VL 2 {E 2 ) . This implies 
that S = S\ + S 2 . At the same time away from equi- 
librium Q(E) > Q(Ei)n(E 2 ) thus S > Si + S 2 [H so 
this entropy becomes non-additive. The crucial differ- 
ence between the d-entropy and 5* = ln(Q(E)) is that we 
do not assume any particular distribution of p nn (E), it is 
determined by the microscopic description. Quite often 
in non-equilibrium situations one associates full entropy 
with the sum of local entropies. The analogue of this 



will be sum of diagonal entropies in the system. Like in 
usual thermodynamics away from equilibrium the differ- 
ence S d — Si d — S 2d might serve as the measure of how 
far the system is away from equilibrium (steady state). 
Careful investigation of this issue requires detailed anal- 
ysis of properties of d-entropy in open systems, which is 
beyond the scope of this work. 

Examples: Let us now illustrate properties of d-entropy 
using a couple of particular examples. First we consider 
a toy-BCS model under repeated quench. This model is 
described by the Hamiltonian: 

n = st-si-^s+s-, (9) 

where Si, 2 = Ylj s i,2(j) is the total spin of N+l spin-1/2 
states, S = Si + S 2 and S ± = S x ± iS y . This Hamil- 
tonian is a simplified version of the BCS Hamiltonian 
in pseudospin representation [l6j |. where single-particle 
spectrum consists of two groups of degenerate energy lev- 
els. The simplicity of the toy-BCS model ((9]) allows us 
to perform large-scale simulations of quantum dynamics 
in this system. The classical dynamics of the BCS model 
was recently studied by several groups to analyze oscil- 
lating superfluidity in BCS superconductors [l^El- Let 
us point that the ground state of the model ([9]) within 
the invariant sector (S z ) — (which reflects the particle- 
hole symmetry in the BCS model) at g < 1 and N 1 
corresponds to a state fully polarized along the z-axis, 
(S§) = —{S*) = N/2. At g = 1 a quantum phase transi- 
tion occurs to the ordered state tilted towards the x — y 
plane with (Sf ) < N/2, in analogy to the transition to 
the superconducting state in the BCS model. 

We calculate the response of the quantum-mechanical 
system ^ to a repeated quench realized by periodically 
changing the coupling g between two values gi and g 2 
both corresponding to the ordered phase (see Fig. 0Ja)). 
The system is initially prepared in its ground state at 
g = <7i = 1.1. Then g is changed instantaneously to the 
value g 2 — 1.2. This results in damped oscillations of the 
magnetization M z {t) = (Sf)/(N + 1) around the mean 
M z (see Fig. [Tfb)). The amplitude of these oscillations 
follows a simple gaussian law 5M Z ~ exp(— £ 2 / T o) with 
a characteristic decay time of the evolution tq — N 1 / 2 . 
At longer times t ~ N there is revival of oscillations due 
to finite size of the Hilbert space. However, if we choose 
large number N 3> 1 then the decay and revival time 
are well separated and the system is in the steady state 
in the interval yN <C t <C N. Once the steady-state 
is reached, the coupling is changed back to its initial 
value g\, and then the process is repeated many times. 
As a result, we find that the average magnetization M z 
decreases with the number of cycles asymptotically ap- 
proaching zero as depicted in FigQ] (see the first fifty 
cycles). We are contrasting the microscopic dynamical 
treatment with the coarse-grained one (red circles), in 
which after each quench after the systems reaches the 
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FIG. 1: The average magnetization M z of the system ((9} with 
N = 2000 states, as a function of the cycle number under re- 
peated quench (see the text for details). After 50 cycles the 
time reversal transformation is applied to the system. The 
blue diamonds correspond to exact time reversibility, black 
squares and green triangles correspond to small time rever- 
sal symmetry breaking perturbations of different magnitudes. 
Red circles represent the coarse-grained dynamics. Inset (a): 
Sketch of a coupling change in the repeated quench. Inset (b): 
Magnetization M z (t) as a function of time after a quench. 



the magnetization, d-entropy, and heat are only weakly 
affected by the time-reversal transformation. We note 
that in a recent work a model similar to ^ sub- 
ject to repeated quench was analyzed. There the authors 
used a different criterion as a measure of irreversibility 
and came to similar conclusions. 
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FIG. 2: Same as in Fig.[T]but for the d-entropy (main graph) 
and heat (inset). 



steady state we artificially set all off-diagonal elements 
of the density matrix to zero. As we discussed earlier, 
this procedure is equivalent to time-averaging of the den- 
sity matrix. Clearly, microscopic and coarse-grained dy- 
namics give close results (see Fig. [T]). Similar trends are 
observed if we analyze time-dependence of the heat and 
the diagonal entropy (see Fig. Both quantities mono- 
tonically increase in time without significant differences 
between the exact and coarse-grained curves. Note that 
as the system heats up the slope of the entropy decreases 
with the cycle number, consistent with general thermo- 
dynamic expectations that AS* ~ AQ/T. 

It is the time-reversed dynamics, which distinguishes 
between full microscopic and coarse-grained descriptions. 
After 50 cycles we perform a time-reversal transformation 
in the system repeating the same sequence of cycles in the 
time-reversed order. All microscopically defined quanti- 
ties including entropy return back to the initial values as 
they should do (see blue diamonds in Figs. [TJ How- 
ever, the coarse-grained dynamics is completely insensi- 
tive to the time-reversal transformation and the system 
continues to heat up. Note that the exact dynamics is 
very sensitive to the presence of small perturbations that 
break the time-reversal symmetry. We analyze this ef- 
fect by slightly changing the coupling g2 — > cj2 = 92 + Sg2 
during only the first cycle after the time-reversal transfor- 
mation with 5g2 = 10 -5 (green triangles) and 5g2 = 10~ 4 
(black squares). We observe that even for very small Sg2 
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FIG. 3: Illustration of the second example: expanding gas. 

Another illustrative example we briefly mention is the 
expanding classical noninteracting gas initially confined 
to the one half of the container separated by the mem- 
brane [2(| (see Fig. [3j. At moment t = the membrane 
is removed and the gas expands to the whole container. 
From the point of view of classical thermodynamics the 
entropy of the gas increases by the amount AS = TV In 2, 
where N is the total number of particles. It is easy to see 
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that the same result applies to the d-entropy. One can 
obtain it by explicit calculation re-expanding the initial 
density matrix in the new basis and computing d-cntropy. 
However, there is a trivial way to see this. As we double 
the volume the density of the momentum states per parti- 
cle doubles, because the momentum is quantized in units 
of 2tt/(L/2) before the expansion and 2w/L after the 
membrane is removed. This means the the same momen- 
tum distribution (momenta of particles do not change in 
this process) is now projected to the twice the number of 
energy states so that by conservation of probability diag- 
onal elements of the density matrix for each particle are 
reduced by a factor of two p nn — > \p nn - Thus we immedi- 
ately see that d-entropy per particle increase by ln(2) so 
that ASd — 7Vln(2). This example also highlights the is- 
sue of non-additivity of the d-entropy away from equilib- 
rium (or more generally steady state). Indeed the jump 
of Sd occurs immediately after the membrane is removed. 
At the same time the local d-entropies corresponding to 
the left and right parts of the container do not change 
right away. We expect that the sum of Sid and S*2d will 
gradually increase in time and after the gas reaches the 
steady state this sum will approach Sd- Then at inter- 
mediate times the difference between Sd and Sid + 
is a possible measure characterizing how far the system 
is away from reaching the steady state. This example 
also highlights importance of quantum mechanics even 
in this purely classical problem. Note that the d-entropy 
changes here purely because we double the number of 
microscopic single particle quantum states. 

Concluding remarks. We showed that d-entropy Sd 
is consistent with properties of the thermodynamic en- 
tropy imposed by the laws of thermodynamics provided 
that initially the system is prepared in thermal equilib- 
rium and then undergoes some dynamical process. The 
d-entropy is microscopically defined and does not violate 
the time-reversal symmetry. In complex closed systems 
for quasi-stationary processes the density matrix always 
remains effectively diagonal and thus as we demonstrated 
Sd should monotonically increase in time. The assump- 
tion of density matrix being always diagonal is similar 
to the assumptions used by Boltzmann in proving the 
famous H-theorem (see e.g. Ref. [2])). However, these 
assumptions can not be justified in general. For exam- 
ple, in simple spin systems one can perform spin-echo 
type experiments, where after quenching magnetic field 
and initial dephasing in the system due to some random- 
ness, one can perform a time-reversal transformation so 
that the spins restore the original coherence. In this case, 
the d-entropy first increases and then decreases back in 
time to the original value. But the situations like this 
where the entropy can decrease in time are very non- 
generic because they require ability to perform a time 
reversal transformation on the Hamiltonian very accu- 
rately (see also discussion in Refs. [H, HH). In Fig. [5] we 
illustrate this point with a toy-model system subject to 



the spin-echo type process. Even a very small perturba- 
tion, which breaks time inversion, completely destroyed 
the reversibility and the d-entropy continues to increase 
in time. From our analysis it follows that the maximum 
entropy state corresponding to p nn independent of n is a 
natural attractor of the Hamiltonian dynamics. In this 
respect, the second law of thermodynamics naturally fol- 
lows from the microscopic equations of motion. 

There is a more subtle issue of relevance of the d- 
entropy to information. If the Hamiltonian is constant in 
time, Sd is only sensitive to the stationary information 
encoded in time-independent diagonal elements of the 
density matrix. Conversely the von Neumann's entropy 
is sensitive to all information in the system, stationary 
or not. This sensitivity results in S n being time indepen- 
dent for any dynamical processes, because time evolution 
is a unitary transformation, which does not change the 
total information content. 

Using Eq. |T]) one can analyze entropy generation in 
various non-equilibrium processes. Increase in Sd can be 
served as a practical measure of non-adiabaticity in the 
system. One can also use the relation T = AE/AS for 
cyclic near equilibrium processes with both AE and AS 
compute independently to determine temperature and 
verify whether this ratio gives a consistent definition of 
temperature independent of the process for steady-state 
non-equilibrium situations. It would be very interesting 
to see if similar general statement can be formulated in 
strongly non-equilibrium situations. 
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help in proving Eq. ([5]) and performing analysis of the 
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S. Girvin, V. Gritsev, V. Gurarie, D. Huse, Y. Kafri, 
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